DSANLS: Accelerating Distributed Nonnegative Matrix Factorization via Sketching Technical Report

Nonnegative matrix factorization (NMF) has been successfully applied in di erent elds, such as text mining, image processing, and video analysis. NMF is the problem of determining two nonnegative low rank matrices U and V , for a given input matrix M , such that M ≈ UV >. There is an increasing interest in parallel and distributed NMF algorithms, due to the high cost of centralized NMF on large matrices. In this paper, we propose a distributed sketched alternating nonnegative least squares (DSANLS) framework for NMF, which utilizes a matrix sketching technique to reduce the size of nonnegative least squares subproblems in each iteration for U and V . We design and analyze two di erent random matrix generation techniques and two subproblem solvers. Our theoretical analysis shows that DSANLS converges to the stationary point of the original NMF problem and it greatly reduces the computational cost in each subproblem as well as the communication cost within the cluster. DSANLS is implemented using MPI for communication, and tested on both dense and sparse real datasets. The results demonstrate the e ciency and scalability of our framework, compared to the state-ofart distributed NMF MPI implementation. Its implementation is available at https://github. com/qianyuqiu79/DSANLS.